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-- The MAILING DATE of this communication appears on the cover sheet with the correspondence address 
Period for Reply 

A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) FROM 
THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1 .136(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

- If the period for reply specified above is less than thirty (30) days, a reply within the statutory minimum of thirty (30) days will be considered timely. 

- If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 1 33). 

- Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment. See 37 CFR 1 .704(b). 

Status 

1 )|El Responsive to communication(s) filed on 25 January 2002 . 
2a)D This action is FINAL. 2b)M This action is non-final. 

3) D Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quayle, 1935 CD. 1 1 , 453 O.G. 213. 

Disposition of Claims 

4) [X] Claim(s) 1-23 is/are pending in the application. 

4a) Of the above claim(s) is/are withdrawn from consideration. 

5) Q Claim(s) is/are allowed. 

6) IEI C!aim(s) 1^22 is/are rejected. 

7) ^ Claim(s) 6,10 and 13 is/are objected to. 

8) D Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

9) E3 The specification is objected to by the Examiner. 

10) [>3 The drawing(s) filed on 25 January 2002 is/are: a)|EI accepted or b)D objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1 .85(a). 
Replacement drawing sheet(s) including the correction is required if the drawing(s) is objected to. See 37 CFR 1.121(d). 

1 1) D The oath or declaration is objected to by the Examiner. Note the attached Office Action or form PTO-152. 
Priority under 35 U.S.C. §§ 119 and 120 

12) D Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 119(a)-(d) or (f). 

a)D All b)Q Some * c)D None of: 

1 0 Certified copies of the priority documents have been received. 

2. Q Certified copies of the priority documents have been received in Application No. . 

3. D Copies of the certified copies of the priority documents have been received in this National Stage 

application from the International Bureau (PCT Rule 17.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 

13) D Acknowledgment is made of a claim for domestic priority under 35 U.S.C. § 119(e) (to a provisional application) 

since a specific reference was included in the first sentence of the specification or in an Application Data Sheet. 
37 CFR 1.78. 

a) □ The translation of the foreign language provisional application has been received. 

14) Q Acknowledgment is made of a claim for domestic priority under 35 U.S.C. §§ 120 and/or 121 since a specific 

reference was included in the first sentence of the specification or in an Application Data Sheet. 37 CFR 1 .78. 



Attachment(s) 

1) £3 Notice of References Cited (PTO-892) 

2) O Notice of Draftsperson's Patent Drawing Review (PTO-948) 

3) [3 Information Disclosure Statement(s) (PTO-1449) Paper No(s) 2 . 



4) □ Interview Summary (PTO-413) Paper No(s). 

5) O Notice of Informal Patent Application (PTO-1 52) 

6) □ Other: 
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PTOL-326 (Rev. 11-03) 
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DETAILED ACTION 



1. This action is responsive to communications of application filed 1/25/2002. 

2. The disposition of the claims is as follows: claims 1-23 are pending in the application. 
Claims 1, 11 and 14 are independent claims. 

3. The group and/or Art Unit location of your application has changed. To aid in the 
correlation of any papers for this application, all further correspondence should be directed to 
Group Art Unit 2676 (effective 1 1/03). Please be sure to use the most current art unit number on 
all correspondence to help us route your case and respond to you in a timely fashion. 



4. The disclosure is objected to because of the following informalities: 

On p. 7, para. [0040], in second sentence "a circle 30 and a tail 32 are modified" should 

be "a circle 36 and a tail 38 are modified". 
Appropriate correction is required. 



5. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the 
basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(a) the invention was known or used by others in this country, or patented or described in a printed publication in this 
or a foreign country, before the invention thereof by the applicant for a patent. 

6. Claims 1-3, 7, 8, 1 1 and 12 are rejected under 35 U.S.C. 102(a) as being disclosed by 
Seni et al., (PGPUB-DOCUMENT-NUMBER: 20030007018), hereafter Seni. 



Specification 



Claim Rejections - 35 USC § 102 
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A. Claim 1, "An image analysis and conversion method comprising: receiving a digital ink 
image [para. 0026]; and converting [para. 0015, 001 6, 0019, 0020] the digital ink image into 
structured object representations of the digital ink image [para. 0024], which are editable by a 
structured text/graphics editor [para. 0022, 0023]" is disclosed [as detailed]. Wherein recognition 
corresponds to converting. 

B. Claim 2, "The method according to claim 1 wherein the converting step includes, altering 
the digital ink image into multiple alternative interpretations" [para. 0024], is disclosed supra for 
claim 1 and [as detailed]. Wherein recognition options correspond to alternative interpretations. 

C. Claim 3, "The method according to claim 2 wherein the altering of the digital ink image 
into multiple alternative interpretations includes, altering the digital ink image into informal 
structured object representations that are editable by the structured text/graphics editor; and 
altering the digital ink image into formal structured object representations that are editable by the 
structured text/graphics editor", is disclosed supra for claim 2 and wherein informal is depicted 
at 104 and formal is depicted under 1 16, "happy" of Fig. 1. 

D. Claim 7, "The method according to claim 1 wherein the step of converting the digital ink 
image to the structured object representations includes generating multiple structured object 
representations of the digital ink image [para. 0016 at "a secondary list of potential recognition 
candidates may be displayed in a box 120"], the multiple structured object representations 
representing at least a first image representation having formal structured object representations 
[para. 0016 at "As each word is recognized, it is shown inserted into the text at the top of the 
interface display 102], and a second image representation containing informal structured object 
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representations [para. 0016 at "A lower portion of the display is designated as a handwriting 
input area 104."] is disclosed supra for claim 1 and [as detailed]. 

E. Claim 8, "The method according to claim 1 wherein the editing by the structured 
text/graphics editor permits movement of structured object representations by at least one of, 
individual objects, a sub-group of all the structured object representations [para. 0022 at "Editing 
operations may include, but are not limited to: insert a space 108, backspace 1 12, delete 1 14, 
capitalize recognition result 110, and undo insertion of last recognition result 106."], or as an 
overall group of the structured object representations" is disclosed supra for claim 1 and [as 
detailed]. Wherein delete corresponds to editing a sub-group of all the structured object 
representations. Said sub-group of all the structured object representations corresponds a letter or 
character of a word or multi-digit number. 

F. Per independent claim 11, this is directed to a system for performing the method of 
independent claim 1, and therefore is rejected to independent claim 1. 

G. Per dependent claim 12, this is directed to a system for performing the method of claim 1 
and in part of dependent claim 7, and therefore is rejected to claim 1 and dependent claim 7. 

Claim Rejections - 35 USC § 103 
7. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 
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8. Claim 4 is rejected under 35 U.S.C. 103(a) as being unpatentable over Seni as applied to 
claim 1 above, and further in view of Official Notice. 

A. Claim 4, "The method according to claim 1 wherein the step of converting the digital ink 
image into structured object representations of the digital ink image includes configuring the 
structured object representations to represent an electronic slide of the structured text/graphics 
editor" is disclosed supra for claim 1 . Although Seni does not appear to disclose, "includes 
configuring the structured object representations to represent an electronic slide", Official notice 
is taken that the art is replete with importing capabilities to configure structured object 
representations to represent an electronic slides for example Powerpoint by Microsoft 
Corporation. 

Therefore it would have been obvious to one of ordinary skill in the art at the time the 
invention was made to apply digital ink recognition disclosed by Seni in combination with 
electronic slides disclosed by Official Notice (Powerpoint), and motivated to combine the 
teachings because it would be obvious since these features are well known in the art. 

9. Claims 5 and 9 are rejected under 35 U.S.C. 103(a) as being unpatentable over Seni as 
applied to claim 1 above, and further in view of Wilcox et aL, (US-PAT-NO: 5,889,523), 
hereafter Wilcox. 

A. Claim 5, "The method according to claim 1, wherein the converting step includes 
forming of an Alternative graph" is disclosed by Seni supra for claim 1. Although Seni does not 
appear to disclose "wherein the converting step includes forming of an Alternative graph", 
Wilcox does in abstract and col. 2, Ins. 24-46. Wherein cluster tree corresponds to alternative 
graph. 
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Therefore it would have been obvious to one of ordinary skill in the art at the time the 
invention was made to apply digital ink recognition disclosed by Seni in combination with 
cluster tree (alternative graph) disclosed by Wilcox, and motivated to combine the teachings 
because it would it is an important interface issue in graphical editing systems as revealed by 
Wilcox in col. 1, lines 12-16. 

B. Claim 9, "The method according to claim 1 wherein the digital ink image is converted 
into the structured objects representations of the digital ink image through the use of an 
Alternative Graph" is disclosed by Seni supra for claim 1 and Wilcox for claim 5. 
10. Claims 14, 15, 17-20, 22 and 23 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Seni as applied to claim 1 above, and further in view of Golovchinsky et al, 
(US-PAT-NO: 6,389,435), hereafter Golovchinsky. 

A. Claim 14, "On a screen display of an electronic device operating a structured 
text/graphics editor, an image representation comprising: structured object representations of a 
digital ink image, the structured object representations correlating to perceptually salient areas of 
the digital ink image, wherein the structured object representations are editable by the structured 
text/graphics editor to allow a user to generate alternative interpretations of the digital ink 
image" is disclosed by Seni supra for claim 1. Although Seni does not appear to disclose 
"representations correlating to perceptually salient areas of the digital ink image", Golovchinsky 
does in [col. 4, Ins. 58-67]. 

Therefore it would have been obvious to one of ordinary skill in the art at the time the 
invention was made to apply digital ink recognition disclosed by Seni in combination with 
salient digital ink disclosed by Golovchinsky, and motivated to combine the teachings because 
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combinations of shapes and colors can be used for the freeform digital ink marks to create a large 
number of identifiable terms as revealed by Golovchinsky in col. 2, lines 23-25. 

B. Claim 15, "The image representation according to claim 14 wherein the structured object 
representations are informal structured object representations, and wherein the informal 
structured object representations are editable to formal structured object representations" is 
disclosed supra for claim 14 and furthermore by Seni in [para. 0022], particularly at "At any 
time, the user can select one (or more) character(s) from a previously entered word and write a 
new character(s) in the input area with the result replacing the selected text". 

Therefore it would have been obvious to one of ordinary skill in the art at the time the 
invention was made to apply digital ink recognition disclosed by Seni in combination with 
salient digital ink disclosed by Golovchinsky, and motivated to combine the teachings because 
combinations of shapes and colors can be used for the freeform digital ink marks to create a large 
number of identifiable terms as revealed by Golovchinsky in col. 2, lines 23-25. 

C. Claim 17, "The image representation according to claim 14 wherein a first structured 
object representation is spatially contained within a second structured object representation" is 
disclosed by Seni and Golovchinsky supra for claim 14. Wherein a letter corresponds to a first 
structured object and a word corresponds to a second structured object wherein the two are 
spatially connected. 

D. Claim 1 8, "The image representation according to claim 14 wherein a new structured 
object representation is added to existing structured object representations" is disclosed by Seni 
and Golovchinsky supra for claim 14. See Seni, Fig. 1, wherein each new word is added to 
existing structured text at top of area 102. 
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E. Claim 19, "The image representation according to claim 14 wherein the structured object 
representations define a text block structure" is disclosed by Seni and Golovchinsky supra for 
claim 14. See Seni, Fig. 1. 

F. Claim 20, "The image representation according to claim 19 wherein the text block 
structure includes a display of text parameters including at least one of text layout, text font, 
bullets, underlines and dummy characters" is disclosed by Seni and Golovchinsky supra for 
claim 18. See Seni, Fig. 1, depicted text layout. 

G. Claim 22, "The image representation according to claim 14 wherein distinct alternative 
interpretations may be displayed at the same time" is disclosed by Seni and Golovchinsky supra 
for claim 14. See Seni, Fig. 1, pop-up window list 120. 

H. Claim 23, "The image representation according to claim 22 wherein display of the 
alternative interpretations is accomplished by the use of at least one of underlays, bubble or 
balloon images, coloring, shading transparency/translucency, defocusing, and pop-up windows" 
is disclosed by Seni and Golovchinsky supra for claim 14. See Seni, Fig. 1, pop-up window list 
120. 

1 1 . Claim 21 is rejected under 35 U.S.C. 103(a) as being unpatentable over Seni as applied to 
claim 1 above, further in view of Golovchinsky et al., (US-PAT-NO: 6,389,435), hereafter 
Golovchinsky, and further in view of Official Notice. 

A. Claim 21, "The image representation according to claim 20 wherein the dummy 
characters are replaceable with target characters" is disclosed by Seni and Golovchinsky supra 
for claim 20. However they do not appear to disclose, "wherein the dummy characters are 
replaceable with target characters", but Official notice is taken that the art is replete wherein the 
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dummy characters are replaceable with target characters in the form of temporary characters, 
templates, boiler plate documents and slides, and place holders. 

Therefore it would have been obvious to one of ordinary skill in the art at the time the 
invention was made to apply digital ink recognition disclosed by Seni in combination with 
salient digital ink disclosed by Golovchinsky coupled with dummy characters disclosed by 
Official Notice (target characters in the form of temporary characters, templates, boiler plate 
documents and slides, and place holders), and motivated to combine the teachings because 
combinations of shapes and colors can be used for the freeform digital ink marks to create a large 
number of identifiable terms as revealed by Golovchinsky in col. 2, lines 23-25. 
12. Claim 16 is rejected under 35 U.S.C. 103(a) as being unpatentable over Seni as applied to 
claim 1 above, further in view of Golovchinsky et al., (US-PAT-NO: 6,389,435), hereafter 
Golovchinsky and further in view of Mahoney et al., (US-PAT-NO: 6,470,095), hereafter 
Mahoney. 

A. Claim 16, "The image representation according to claim 15 wherein the alternative 
interpretations permit a mixing of formal structured object representations and informal 
structured object representations in a single image representation displayed on the computer 
screen" is disclosed by Seni and Golovchinsky supra for claim 15. Although Seni and 
Golovchinsky do not appear to disclose, "wherein the alternative interpretations permit a mixing 
of formal structured object representations and informal structured object representations in a 
single image representation displayed on the computer screen", Mahoney does in Fig. 3 at col. 2, 
Ins. 45-49. 
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Therefore it would have been obvious to one of ordinary skill in the art at the time the 
invention was made to apply digital ink recognition disclosed by Seni in combination with 
salient digital ink disclosed by Golovchinsky coupled with handwritten and typeset text 
displayed in a single image disclosed by Mahoney, and motivated to combine the teachings 
because combinations of shapes and colors can be used for the freeform digital ink marks to 
create a large number of identifiable terms as revealed by Golovchinsky in col. 2, lines 23-25 and 
because the amount of a human user's time required to enter and edit such user-created borders is 
significant as revealed by Mahoney in col. 1, Ins. 22-23. 



17. Claims 6, 10 and 13 are objected to as being dependent upon a rejected base claim, but 
would be allowable if rewritten in independent form including all of the limitations of the base 
claim and any intervening claims. 



18. Responses to this action should be mailed to: Commissioner of Patents and Trademarks, 
Washington, D.C. 20231. If applicant desires to fax a response, (703) 308-9051 may be used for 
formal communications or (703) 308-6606 for informal or draft communications. 

Please label "PROPOSED" or "DRAFT" for informal facsimile communications. Hand- 
delivered responses should be brought to Crystal Park II, 2121 Crystal Drive, Arlington, VA., 
Sixth Floor (Receptionist). 



Allowable Subject Matter 



Responses 



# 
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Inquiries 



19. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Greg Cunningham whose telephone number is (703) 308-6109. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Matthew Bella, can be reached on (703) 308-6829. 

Any response to this action should be mailed to: 



Hand-delivered responses should be brought to Crystal Park II, 2121 Crystal Drive, 
Arlington, VA, Sixth Floor (Receptionist). 

Any inquiry of a general nature or relating to the status of this application or proceeding 
should be directed to the Technology Center 2600 Customer Service Office whose telephone 
number is (703) 306-0377. 



Commissioner of Patents and Trademarks 



Washington, D.C. 20231 



or faxed to: 



(703) 872-9306 (for Technology Center 2600 only) 




gfc 




November 14, 2003 



MATTHEW C. BELLA 
SUPERVISORY PATENT EXAMINER 
TECHNOLOGY CENTER 2600 



